Self-Report Measures of Intelligence: Are They Useful as Proxy IQ Tests?

Correlations between single-item self-reports of intelligence and IQ scores are rather low (.20–.25) in college samples. The literature suggested that self-reports could be improved by three strategies: (1) aggregation, (2) item weighting, and (3) use of indirect, rather than direct, questions. To evaluate these strategies, we compared the validity of aggregated and unaggregated versions of direct measures with four indirect measures (Gough’s Intellectual efficiency scale, Hogan’s Intellect composite scale, Sternberg’s Behavior Check List, and Trapnell’s Smart scale). All measures were administered to two large samples of undergraduates (Ns = 310, 326), who also took an IQ test. Although results showed some success for both direct and indirect measures, the failure of their validities to exceed .30 impugns their utility as IQ proxies in competitive college samples. The content of the most valid items referred to global mental abilities or reading involvement. Aggregation benefited indirect more than direct measures, but prototype-weighting contributed little. Journal of Personality 66:4, August 1998. Copyright © 1998 by Blackwell Publishers, 350 Main Street, Malden, MA 02148, USA, and 108 Cowley Road, Oxford, OX4 1JF, UK. This research was supported by a grant to the senior author from the Social Sciences and Humanities Research Council of Canada. We wish to thank Harrison Gough, Robert Hogan, Robert Sternberg, Paul Trapnell, and several anonymous reviewers for comments on an earlier draft. Correspondence should be addressed to Del Paulhus, Department of Psychology, University of British Columbia, Vancouver, Canada, V6T 1Z4. Electronic mail may be sent via the Internet to [email protected] Can people validly rate their own intelligence? Skeptics argue that such self-reports are hopelessly contaminated with a variety of distortions including self-deception, impression management, and reconstrual. Such defensive reactions may explain the low variance in self-ratings of intelligence: Rarely do people rate themselves as “below average” (McCrae, 1990). Even the most forthright and insightful individuals, skeptics warn, can never confirm the veracity of their self-assessments because the concept itself is so elusive in nature. Despite its elusiveness, the concept of intelligence plays a central role in psychological research, particularly in such contexts as educational evaluation, personnel selection, and child development. To facilitate such research, considerable effort has been devoted to developing self-report alternatives to cumbersome IQ tests of intelligence. Such approaches have progressed well beyond a simple request to “rate how intelligent you are.” Three strategies, in particular, have been recommended. One is the use of indirect assessment to bypass the inevitable defensiveness of a direct request for a self-rating. A second, the aggregation strategy, favors multiple-item over single-item measures. A third strategy, prototype weighting, takes into account the differential importance of items within a measure. In this report we evaluate these three strategies by determining their ability to improve prediction of performance on IQ tests. Use of IQ tests as the criterion for intelligence ratings has not yielded high validities, particularly in college samples. The validities are somewhat higher for observer-ratings than for self-ratings. Values in the range of .25 to .50 have been found when the judgment is made by spouses (Bailey & Mettetal, 1977), by friends and strangers (Borkenau, 1993), by adolescent acquaintances (Bailey & Hatch, 1979), and by long-term discussion-group colleagues (Paulhus & Morgan, 1997). Except in the case of spouses, however, the achievement of these solid validities required aggregation across multiple observers. Self-perceptions typically parallel other-perceptions but, to the extent that the trait being evaluated is highly evaluative (e.g., intelligence), the former are noticeably less valid (John & Robins, 1993). Studies using IQ test scores as the criterion have yielded single-item validities of .32 (Borkenau & Liebler, 1993) and .38 (Reynolds & Gifford, 1996) in 1. The term “validity” is used to mean correlation with a specific criterion. Its use does not imply that IQ is the sole criterion for measuring intelligence. 526 Paulhus et al.